Bayesian Parameter Estimation
This page contains resources about Bayesian Parameter Estimation, Bayesian Parameter Learning and Bayes Estimator. Subfields and Concepts * For complete (fully observed data): ** Dirichlet distribution (or other priors) * For incomplete (hidden/missing data): ** Markov Chain Monte Carlo (MCMC) ** Viterbi Algorithm ** Variational Bayes *** Stochastic Gradient Variational Bayes (SGVB) Estimator * Bayesian Hierarchical Modelling / Hierarchical Bayes Model ** Hyperparameter ** Hyperprior * Bayesian Decision Theory * Bayesian Point Estimation * Bayesian Signal Processing * Bayes Risk * Bayesian Score ** Posterior variance (when MSE is used) * Bayes Risk Function / Posterior Expected Loss (i.e. Posterior Expectation Value of Loss Function) ** Posterior mean / Minimum MSE (MMSE) estimator / Bayes least squared error (BLSE) estimator / Squared error loss ** Posterior median / Median-unbiased estimator / Absolute error loss ** Posterior mode * Bayes estimator ** MMSE / BLSE estimator ** Median-unbiased estimator ** Bayes estimator for conjugate priors (eg. exponential family) * Maximum Likelihood Estimation (MLE) * Asymptotics of Maximum Likelihood * Cramer-Rao bound / Cramer-Rao lower bound * Fisher information * Uninformative priors / Noninformative priors / Maximum entropy priors ** Jeffreys prior * Maximum Entropy (Maxent) Models / Entropic priors * Exponential family * Beta distribution * Bayesian Online Parameter Estimation * Recursive Bayesian Estimation / Bayes filter ** Kalman filter (special case of Bayes filter) ** Wiener filter (special case of Kalman filter) * Bayesian Density Estimation * Nonparametric Empirical Bayes (NPEB) * Parametric Empirical Bayes Point Estimation * Iterative proportional fitting (IPF) * Nonparametric Methods ** Kernel Density Estimation (KDE) ** k-Nearest Neighbours ** Bayesian Nonparametrics Books and Book Chapters * Theodoridis, S. (2015). "Chapter 12: Bayesian Learning" Machine Learning: A Bayesian and Optimization Perspective. Academic Press. * Aster, R. C., Borchers, B., & Thurber, C. (2012). "Chapter 11: Bayesian Methods". Parameter estimation and inverse problems. Academic Press * Barber, D. (2012). "Section 9.1: Learning as Inference". Bayesian Reasoning and Machine Learning. Cambridge University Press. * Barber, D. (2012). "Chapter 18: Bayesian Linear Models". Bayesian Reasoning and Machine Learning. Cambridge University Press. * Duda, R. O., Hart, P. E., & Stork, D. G. (2012). "Chapter 2: Bayesian Decision Theory". Pattern Classification. John Wiley & Sons. * Murphy, K. P. (2012). "Section 5.7: Bayesian Decision Theory". Machine Learning: A Probabilistic Perspective. MIT Press. * Koller, D., & Friedman, N. (2009). "Chapter 17: Parameter Estimation". Probabilistic Graphical Models. MIT Press. * Theodoridis, S., Pikrakis, A., Koutroumbas, K., & Cavouras, D. (2008). "Chapter 2: Bayesian Decision Theory". Pattern Recognition. 4th Ed. Academic Press. * Bishop, C. M. (2006). "Chapter 2: Probability Distributions". Pattern Recognition and Machine Learning. Springer. * MacKay, D. J. (2003). "Chapter 24: Exact Marginalization". Information Theory, Inference and Learning Algorithms. Cambridge University Press. * MacKay, D. J. (2003). "Chapter 36: Decision Theory". Information Theory, Inference and Learning Algorithms. Cambridge University Press. * Bretthorst, G. L. (1998). Bayesian spectrum analysis and parameter estimation. Springer Science & Business Media. * Berger, J. O. (1993). Statistical decision theory and Bayesian analysis. 2nd Ed. Springer Science & Business Media. Scholarly Articles * Caticha, A. (2010). Entropic inference. arXiv preprint arXiv:1011.0723. * Caticha, A., & Preuss, R. (2004). Maximum entropy and Bayesian data analysis: Entropic prior distributions. Physical Review E, 70(4), 046127. * Ghahramani, Z. (2003). "Graphical models: Parameter learning". In Handbook of Brain Theory and Neural Networks. * Malouf, R. (2002). A comparison of algorithms for maximum entropy parameter estimation. In proceedings of the 6th conference on Natural language learning-Volume 20 (pp. 1-7). Association for Computational Linguistics. See also * Bayesian Model Selection * Estimation Theory Other resources * Bayesian parameter estimation - Metacademy * Maximum Entropy Modelling Category:Bayesian Machine Learning